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Foreword 


ENQA realises that there is a steadily growing interest among students and all 
stakeholders of Higher Education and the public at large in accessing detailed and 
reliable information on the quality of individual study programmes, faculties and 
higher education institutions. ENQA is aware that external quality assurance activities 
in higher education set up by quality assurance agencies are producing only part of the 
information. Besides quality assurance agencies that enshrine their activities in the 
ESG, there is a growing number of providers of information about higher education. 
The proliferation of ranking schemes and other transparency tools is one aspect of this 
increasing need for information about higher education institutions and their activities. 

The relevance of ranking and other transparency tools has recently been echoed 
by the Leuven/Louvain-La-Neuve Communique of 2009. In the ongoing debate about 
purpose and relevance, about strengths and threats of rankings ENQA raised the 
concern that transparency tools could foil the enhancement-driven aspect of quality 
assurance in higher education by stifling self-critical reflection. 

The purpose of this seminar was to discuss expectations from higher education 
institutions and other stakeholders concerning information on quality of higher 
education, and the contribution of quality assurance and transparency tools to meet 
these expectations. The outcome of the discussions is a position paper on information 
and transparency tools, which can be found in Part 2 of this report. 


Achim Hopbach 
President 

European Association for Quality Assurance in Higher Education (ENQA) 


Part 1 - Key messages 

The main conclusions identified in the present report and during the discussions at the 
seminar are outlined here below. 


1.1 Key messages from Session 1: 

Higher Education and Quality: Expectations, Quality Standards and Tools. 
What kind of information are we looking for? And for what purpose? 

Padraig Walsh, Irish Universities Quality Board (IUQB) 

The European University Association (EUA), the European Association of Institutions 
in Higher Education (EURASHE), the University of Bologna, the European Students' 
Union (ESU), BusinessEurope and the Bologna Follow-up Group (BFUG) contributed to 
this session, chaired by Padraig Walsh. What messages are there for ENQA arising from 
the presentations in this session? 

1. There has to be greater linkage between the internal QA processes and 
associated information provided by HEIs and the external QA processes and 
associated information provided by QA agencies. ENQA and its member agencies 
need to have positive interaction with HEIs to improve this linkage. 

2. Currently, information on external QA (published reports etc.) is better (largely 
because it is the major product of QA agencies) than information on internal QA 
which is often inaccessible (because it is a by-product of the main “products” of 
HEIs: programmes and graduates). 

3. Transparency tools are not the same as rankings but the phrase is being 
used synonymously. ENQA and member agencies should not use the phrase 
“transparency tools” as a form of polite evasion of using the phrase “rankings”. 

4. Transparency and quality assurance are not synonymous either. Transparency 
should be a desirable outcome of good QA processes. 

5. QA must be as much about communication as information. ENQA has a role 
to play in communicating to stakeholders where they can find the relevant 
information and how to interpret it. The level of communication needs to be 
increased rather than just providing more information. 

6. Students need to be involved in the development of internal and external QA 

processes. They also need to be consulted as to what information (which must be 
honest and true) they need to make informed decisions as independent citizens. 
ENQA can assist by working with students to determine who can best provide this 
information (the balance between HEIs, QA agencies and other bodies) and how 
best to communicate the information. 


7. Employers are interested in information about the efficiency of HEIs (graduation 
rates, drop-out rates, employability rates and about information on student 
learning outcomes. ENQA can assist by ensuring that external QA processes 
conducted by agencies verify the data arising from these input and output 
measures (ensure that ESG 2.1 verifies ESG 1.3 and 1.7). 

8. People need information in the reliability of QA agencies and whether they are 
functioning according to ESG. ENQA needs to communicate the significance of 
full membership following robust external review. The credibility of ENQA and 
EQAR will be significantly enhanced only when full member agencies and listed 
agencies respectively encompass the vast majority of the 47 member countries of 
the EHQA (up from the present 53% and 19% respectively). 

1.2 Key messages from Session 2: 

Higher Education and Quality: Expectations, Quality Standards and Tools. 
What kind of information do we provide? And for what purpose? 

Helka Kekalainen, Finnish Higher Education Evaluation Council (FINHEEC) 

The European Association for Quality Assurance in Higher Education (ENQA), the 
Hungarian Accreditation Committee (HAC), the Finnish Higher Education Evaluation 
Council (FINHEEC) and the European University Association (EUA) contributed to 
this session, chaired by Patrick Van den Bosch, Flemish Interuniversity Council (VLIR). 
What messages are there for ENQA arising from the presentations in this session? 

In the opening speech for the session Karl Dittrich, representing ENQA, called for 
even greater ambition to deliver demonstrable quality and to make this quality visible 
in an objective and independent manner. European Union’s strategy has been to show 
the diversity that exists within the European Higher Education Area (EHEA). ENQA is 
very much aware that the diversity between its member organisations is very large. 

The most important role ENQA has to play in providing information on the quality 
of Higher Education lies with the membership of ENQA itself. By cyclical external 
evaluation against ESG, the membership organisations demonstrate that they meet 
the rules for organisation and work method: the criteria of independence, soundness, 
openness, transparency and effectiveness. In this way, they also give legitimacy to the 
assessments that they issue. 

Is the information provided sufficient? The existing instruments such as the diploma 
supplement, the ECTS records and the learning outcomes are not yet sufficient. 
Moreover, there is increasing national and European political pressure to produce 
sharper pictures of the quality of institutions and programmes which are, for example, 
focused on “excellence” or on real content-based internationalisation. 

There are a number of serious obstacles for better information: 

1. Emphasis on accountability calls for a dichotomous assessment. 

2. Usefulness for providing information suffers, when assessments must meet strict 
procedural and form requirements in order to avoid appeals or objections. 

3. Higher education institutions are relatively hesitant to be tested in an independent 
manner. 

4. A wide range of legal rules surrounding privacy hinder a free provision of 
information. 


Dittrich concluded by saying that the establishment and existence of ENQA is having 
a legitimising effect on the system of external quality assurance. ENQA thus indirectly 
contributes to providing reliable information for the stakeholders. The quality and the 
content of this information are determined to a significant degree by the national laws 
and regulations, but also by the behaviour of the educational institutions themselves. 
ENQA is open to all suggestions for increasing the quality of the information provided 
for the benefit of the stakeholders. 

Two agency perspectives were given by Gyorgy Bazsa, HAC and Helka Kekalainen, 
FINHEEC. Both presentations answered the questions: What kind of information do 
we provide? And for what purpose? 

The functions of HAC in the Hungarian higher education quality system are external 
evaluations and accreditations. By law the HEIs operate internal QA systems. HEIs 
are obligated to prepare quality development schemes, which specify the processes 
of operation and assessment. The HEIs have to annually revise the implementation 
of the quality development schemes and revisit the action plans. Findings should be 
published on the website. HAC’s resolutions are legally expert opinions. They have 
ayes or no outcome with argumentation. They are published on the website. Also 
the resolutions of the independent appeal board are public. The Hungarian higher 
education institutions carry out intensive, broad and competitive PR activity to attract 
new students. 

Institutional accreditations address mission, management, research, facilities and 
internal QA systems and the information is mainly directed towards the institution and 
the larger society. Programme accreditations serve more students, staff and the labour 
market by looking at aim/mission, curriculum, study structure, teaching/learning 
methods, assessment of students, teaching staff, research and internal QA. The main 
purposes of HAC’s activities are public transparency, confidence in Hungarian higher 
education system and ENQA membership. 

The Finnish institutional Audit of QA System aims at evaluating what procedures 
and processes the HEI uses to maintain and develop the quality of its education and 
other activities; whether the HEI’s quality assurance works as intended, whether 
the QA system produces useful and relevant information for the improvement of its 
operations, and whether it brings about effective improvement measures. The audit 
addresses the comprehensiveness, effectiveness and transparency of the QA system. 

In the Finnish Centres of Excellence in University Education -evaluation, the content 
areas and criteria are the mission of the unit, programme and course design, delivery of 
education, outputs and continual development. 

FINHEEC’s tools provide HEIs with QA system check, recommendations for 
improvements, good practises, benchmark opportunities and acknowledgement for 
excellence in education. 

For policy-makers, FINHEEC offers evaluation of each HEI’s QA system, thematic 
system-wide analysis and bases for granting performance based funding. Students may 
gain a guarantee that the institution has a functioning QA System and some knowledge 
about it, as well as, an idea of a department that has a strong emphasis on teaching and 
education (Centre of Excellence in Education). Employers and other business partners 
receive knowledge of each HEI’s QA system, national system and Centres of Excellence 
in Education. 


Tia Loukkola, EUA, answered the questions on behalf of the European universities. 
She began by saying that it is a challenging question since EUA has not done real 
research on the topic. 

All universities provide some kind of marketing material, namely brochures of the 
programmes they offer for the large public, their partners and students. Universities 
need to meet the needs of other potential institutions/partners. Universities are 
reporting to their local governments and ministries on their activities following their 
national/regional schemes/instructions. They provide ad hoc information on their 
activities as well. 

EUA has worked on promoting transparency and sharing information through 
projects. EUA has formulated several policies on quality and quality assurance: 
there should be a balance between accountability and enhancement. EUA is trying 
to encourage universities to be more open and communicate more with external 
stakeholders. Universities should have greater courage to express their diverse missions 
and goals, and to get tested in the light of these goals. Research universities seem to 
get more attention and prestige. Universities might be tempted to profile themselves as 
research institutions. Loukkola concluded by discussing some preliminary ideas from 
an ongoing project on quality culture within institutions. 

1.3 Key messages from Session 3: 

Higher Education and Quality: Expectations, Quality Standards and Tools. 
What kind of information do we provide? And for what purpose? 

The European Commission, the classification project (CEIHE 1 ) and the Centre for 
Higher Education Development (CHE) contributed to this session, chaired by Robin 
van IJperen. 

Introduction 

Robin van IJperen, European Commission 

The title of the seminar, in particular on the link to institutional autonomy, is the 
core of the EU modernisation agenda for universities. An important message from this 
modernisation agenda, next to the call for more autonomy, is that institutions face 
increasing competition in this globalising higher education area. The Commission 
therefore thinks that institutions should develop a clearer profile on the basis of their 
strengths, where they can distinguish themselves. Within this profile institutions 
should seek for partners at national and international level (universities, enterprises) to 
further improve their strengths. 

Institutions should communicate clearly to the different stakeholders on their 
profile and performances and this is, where also quality assurance comes in. As quality 
assurance evaluations give a wealth of information. However, the problem is that 
quality assurance is lacking transparency and comparability at the European level, 
as the Commission concludes in its recent Progress Report on Quality Assurance in 
Higher Education. This probably explains the current success of rankings, which can 
give a simplified comparative picture of quality, based on some indicators. This point 
has been recognised in one of the ENQA documents for this seminar, stating that there 


1 Classifying European Institutions for Higher Education (CEIHE) 


is a fear in the quality assurance community that rankings, the strive of institutions for 
a high position in the most famous rankings in particular, will replace their self-critical 
assessment and efforts towards quality enhancement. 

The European Commission believes that a certain “rat race” for a high ranking 
position is unavoidable, but a high ranking position should not be the only goal for 
institutions. It is more important that institutions use rankings in a strategic way, in 
the sense that institutions analyse the qualitative background of their scores in order 
to indeed improve themselves and enhance their quality. In other words, rankings can 
give a simple comparative picture of the main differences in performances, which could 
be used as starting point for a deeper analysis of the strengths and weaknesses of an 
institution, for example by using information coming from quality assurance reviews. 

The European Commission is aware of the main criticism on existing rankings, for 
example that they focus only on research performance and ignore performance in other 
areas, such as teaching quality. On the other hand, rankings are here to stay and feed 
the increasing need for transparency. The European Commission believes it is better 
to join the debate on transparency, and therefore supports the development of more 
sophisticated instruments, for example via the EU-supported feasibility study on the 
design of a multi-dimensional global ranking, which will be presented in this workshop. 

Rather than fiercely going against any use of the ranking instrument, this workshop 
should focus on the challenge of how the strengths and weaknesses of both quality 
assurance instruments and rankings could be combined and/or complement each other. 

What messages arose from the presentations in this session? 

Nathalie Costes, ENQA Secretariat 

The key messages from this session were the following: 

1. Higher education systems in Europe are characterised by diversity. Higher 
education institutions are diverse in their function and scope. Classifications 
describe horizontal diversity and provide institutional profiles while rankings 
assess vertical diversity within types of institutions and result in performance- 
based outcome. Classifications and rankings are complementary instruments of 
transparency to map diversity and to satisfy the wish, expressed at the political 
level, to increase ‘transparency of diversity’. 

2. Classifications are instruments to group higher education institutions and to 
characterise similarities and differences based on actual conditions and activities 
of institutions, not on performance. Classifications are used as a transparency 
tool for various stakeholders: as an instrument for institutional strategies (i.e. 
to help institutions develop their mission, profile and association with partners); 
as a base for governmental policies better targeted to institutions; as a tool for 
research; and as an instrument for better ranking. 

3. A common point between classification and quality assurance could be that, 
in both cases, institutions are looked at. Rankings are market oriented. Unlike 
quality assessment, rankings ignore the causes of programme/institutional 
weaknesses. They refer to a method, which consists in comparing and rating 


institutions by numeric indicators, without analysing the processes. 

4. In U-Map and U-Multirank projects, classification is a precondition for ranking. 
Focused institutional rankings compare the performance of institutions along 
one of the six dimensions of the classification (teaching and learning profile; 
student profile; research; knowledge exchange; internationalisation and regional 
engagement). Field-based rankings (i.e. rankings of study programmes) produce 
performance profiles of specific fields in institutions which have comparable 
profiles. Therefore, field-based rankings take into consideration the multi- 
dimensional classification of entire institutions. 

5. There is no one-size-fits-all-approach for rankings. Rankings should have a 
clear purpose and target group (Berlin principles), with which indicators should 
be in line. Rankings should compare equivalent programmes and institutions 

in terms of mission and profile. International rankings should particularly care 
about comparing comparable institutions, but comparable data are not always 
available. 

6. The significance of rankings depends on the way indicators are measured and 
higher education institutions compared. CHE offers an alternative approach 
to rankings. According to CHE, rankings should be subject-specific and not 
institution-based. They should be multi-dimensional and give a detailed analysis 
of each indicator in order to better differentiate the strengths and weaknesses 

of universities and to provide relevant information for specific target groups. 
Rankings should adopt a group approach (top group, middle group, bottom 
group) rather than listing institutions in single league tables. 


Part 2 - ENQA Position Paper on 
Transparency Tools 

(Adopted on 4 March 2011) 

Executive summary 

Higher Education Institutions (HEIs) face a steadily growing interest among students, 
all stakeholders and the public at large in accessing detailed and reliable information 
on individual study programmes, faculties and higher education institutions, and 
especially on quality at programme and at institutional levels. 

A closer look at this steadily growing interest demonstrates that the interest covers a 
wide spectrum of specific interests which depend on the differing perspective of each 
and every stakeholder, students and public authorities in particular. 

Consequently, it would be misleading to assume a shared view on what is relevant 
information about higher education and to assume a common understanding of role 
and meaning of ‘reliable information’ and ‘transparency’. Transparency cannot be 
considered mono-dimensional. The differentiation relates to the purpose for which 
the information is requested, which, consequently, determines the type of information 
needed. In principle, information is requested for the following purposes: 

• to choose a programme; 

• to choose an institution; 

• to make strategic decisions at institutional and system levels; 

• to provide confidence in the output of HEIs. 

Therefore, the following types of information can be distinguished: 

• Information about the profile and quality of programmes and qualifications; 

• Information about the performance and potential of institutions; 

• Information about the overall performance and potential of higher education 
systems. 

In particular, the distinction between performance and potential as two features of the 
quality is relevant as regards the type and source of information. 

Transparency tools and their contribution in informing about HE 

THE BOLOGNA TRANSPARENCY TOOLS 

It is noteworthy that transparency in higher education and tools for creating 
transparency are not new issues in the discussion about moving the Bologna Process 
further. Quite the contrary, the Bologna Process may even be named the most 
significant factor in striving for transparency in European higher education, in 
particular by introducing transparency tools such as Qualifications Frameworks, 
Diploma Supplement and ECTS. 

RANKINGS 

Rankings list certain groups of HEIs that are ranked comparatively according 
to common numerical performance indicators. In doing so, rankings provide a 


comparative view at the system level, which helps understanding the performance of a 
whole HE system and of single institutions compared to others. 

The project called Classifying European Institutions for Higher Education (CEIHE) 
is developing a tool to describe the increasing diversity in European higher education. 
This tool will enable stakeholders to see the various missions and profiles of HEIs. The 
classification will be multi-dimensional, non-hierarchical and descriptive instead of 
prescriptive. 

QUALITY ASSURANCE PROCEDURES 

External quality assurance procedures may cover a wide spectrum of processes 
designed to monitor, maintain and enhance quality. In principle, quality assurance 
serves two main purposes: enhancement and accountability. Two major features of the 
type of information provided by quality assurance are: 

• The link between processes and performance/quality; 

• The link between evaluation of quality and performance (including weaknesses 
and strengths) and recommendations for future actions. 

Thus, quality assurance does provide comprehensive information about strengths and 
weaknesses and thus, a holistic view on the quality of a programme or an institution. It 
also draws attention to potential future enhancement. 

CONCLUSION 

Bologna transparency tools, quality assurance and rankings/classifications serve 
different purposes, although they support each other to a certain extent. It is 
misleading to consider them as alternatives. From the view point of quality assurance 
and its two main purposes, accountability and enhancement, their relationship to other 
transparency tools looks as follows: 

The Bologna transparency tools are important reference points for quality assurance. 
The qualification framework for the European Higher Education Area describes level 
and scope of qualifications graduates will have acquired by the time they graduate. The 
diploma supplement describes the profile of a qualification, and ECTS is an important 
tool to guarantee realistic curriculum design as regards student workload, and also 
fosters mobility. 

Rankings and classifications, by putting performance of institutions in relation to 
criteria, contribute to the accountability function by informing the public, but they 
do not contribute directly to quality enhancement, the second function of quality 
assurance. Thus, rankings and classification tools should not be seen as quality 
assurance tools; in particular, they do not provide information about the potential 
for the future, although HEIs may draw conclusions from rankings. They might be 
seen rather as providers of a certain type of information that is useful for quality 
assurance. On the other hand, it is true that quality assurance may provide quantitative 
information on aspects of the performance of a programme or an institution for 
comparison purposes, although this is not the core purpose. 


ENQA Position Paper on 
Transparency Tools 


Introduction 

Higher Education Institutions (HEIs) face a steadily growing interest among students, 
all stakeholders and the public at large in accessing detailed and reliable information 
on individual study programmes, faculties and higher education institutions, and 
especially on the quality at programme and at institutional levels. 

Also, in the political arena, information on higher education has reached the top end of 
the agenda, not least triggered by the debate about whether rankings contribute to the 
public call for reliable information. Even within the frame of the Bologna process, the 
issue of rankings was taken up. However, in the Leuven/Louvain-la-Neuve communique 
of 2009, the concept of transparency tools was introduced instead to avoid controversy. 
The purpose of this position paper is to examine the nature and purpose of such 
transparency tools and, in particular, their relationship to quality assurance. The 
paper is based on discussions during a workshop organised by ENQA and the Magna 
Charta Observatory in Bologna, Italy, to discuss the meaning of quality assurance and 
transparency tools among relevant stakeholders and actors in the fields of both quality 
assurance and transparency tools. 

1. Information about higher education: Who asks for transparent information 
and what for? 

Not surprisingly, a closer look to the steadily growing interest in detailed and reliable 
information on higher education demonstrates that the interest covers a wide spectrum 
of specific interests which depend on the specific perspective of each and every 
stakeholder. Consequently, deliberations about the request for “information” and how 
to respond to this request best need to start with answering the question: Who asks for 
transparent information and what for? 

In the following, the typical interests of various stakeholders shall be summarised 
and represented pointedly for the sake of better understanding the different 
perspectives: 

1.1 WHO ASKS FOR TRANSPARENT INFORMATION? 

Students 

In times of growing student mobility on the one hand, and of growing diversification of 
institutions and programmes on the other, students ask for reliable information to help 
them decide which programme meets their expectations best. In particular, students 
ask for information about the qualification level and profile of a certain programme, 
and information about learning conditions in a broad sense, which includes academic 
and social aspects. In addition, students are interested in reliable information about the 
quality of the programme and its recognition status. 


Higher education institutions 

HEIs are becoming more and more autonomous as regards their own strategy, profile, 
and delivery of programmes. At the same time, institutions act in a more and more 
competitive world. Performance based funding schemes combined - in some countries 
- with shrinking HE investments, mobile students, and the growing expectations with 
regard to society, widening access etc. mean that HEIs compete for funding, students 
and not least reputation. Thus, HEIs seek reliable information on their performance 
in order to better inform their strategic and operational decisions concerning future 
developments. 

Employers 

The employers’ interest in reliable information is mainly twofold. On the one hand, 
they need reliable information about programmes and qualifications to better 
understand the skills and competencies obtained by graduates. On the other hand, they 
look for information about profile, experience, and reputation of institutions in the 
field of research and development, as well as in the field of education and training for 
choosing a partner for collaborative projects and strategic partnerships. 

Public authorities and political decision makers 

By nature, public authorities and political decision makers have a very broad interest in 
reliable information about the whole higher education system. On the one hand, this 
serves as basis for better informed political decisions, and on the other, it is necessary 
to evaluate the information about whether the political aims, such as raising the level 
of skills and competencies, raising the quality, and consequently, the attractiveness 
of programmes and institutions for international students have been achieved. 

Political decision makers in many countries are particularly interested in institutional 
performance because a substantial part of institutional funding is allocated according 
to performance indicators. 

1.2 What lor? 

PURPOSE OF INFORMATION 

This short overview clearly demonstrates that it would be misleading to assume a 
shared view on what is relevant information about higher education and to assume 
a common understanding of the role and meaning of ‘reliable information’ and 
‘transparency’. Transparency can’t be considered mono-dimensional. The differentiation 
relates to the purpose for which the information is requested, which, consequently, 
determines the type of information. 

In principle, information is requested for the following purposes: 

• to choose a programme; 

• to choose an institution; 

• to make strategic decisions at institutional and system levels; 

• to provide confidence in the output of HEIs. 

Therefore, the following types of information can be distinguished: 

• Information about the profile and quality of programmes and qualifications 

• Information about the performance and potentials of institutions 


• Information about the overall performance and potentials of higher education 
systems 

In particular, the distinction between performance and potentials as two features of the 
quality is relevant as regards the type and source of information. 

The different purposes of collecting reliable information are a consequence of 
three major, interrelated developmental trends in higher education : competition, 
diversification and autonomy. 

COMPETITION 

Competition has become a major driving force in higher education. HEIs increasingly 
compete for funds, students and reputation. 

Performance based funding schemes, increasing proportions of third party funding 
and increasing student mobility (especially at master and PhD level) mean that HEIs 
have to demonstrate performance and quality. 

DIVERSIFICATION 

Diversification is threefold: Firstly, expectations from students change, simply 
because the student population is getting more and more diverse. The male, young, 
full-time student, coming directly from a local or national secondary education 
institution is no longer a typical student. Educational, social, cultural, linguistic and 
national backgrounds are getting more and more diverse, and so do the requests for 
appropriate study programmes, such as part-time programmes, E-learning programmes, 
programmes in the frame of lifelong learning, etc. Thus, the mode of provision changes 
in terms of organisation and structure. 

Secondly, employers' expectations change: The qualifications, competencies and 
skills that are required of graduates by employers are developing and diversifying 
increasingly quickly. This also leads to conflicts with the educational mission of HEIs. 

Thirdly, the expectations from society at large change. HEIs are not only requested 
to provide academic education to students and to carry out blue skies or applied 
research. They are also requested to contribute significantly to meeting the challenges 
of society and mankind, ranging from social and financial welfare and cultural 
development to ecological questions such as global warming. 

These developments lead to the diversification of the mission, profile and strategy of 
HEIs, not forgetting their provision of teaching. 

AUTONOMY OF HEIs 

The third major developmental trend, that is growing the autonomy of higher education 
institutions, is closely interrelated with completion and diversification. Alongside 
with substantial administrative reforms in all public sectors since the 1970s, higher 
education institutions were given more autonomy and responsibilities in terms of 
management of resources, personnel, and of academic decisions in teaching and 
research. However, the independence from state regulation went hand in hand with 
the necessity to account for the quality (i.e. the effectiveness and efficiency) of their 
services via reporting systems and quality assurance. 

In a diversified and competitive higher education system with HEIs as autonomous 
actors, it is of utmost importance to have detailed and reliable information to help 


make well-informed decisions, be it about the institution that a prospective student 
wants to enrol at, be it about funding according to performance indicators, or be it 
about a suitable partner for a joint research project. In principle, the purposes for 
providing information can be summarised under two headings: 

INFORMATION FOR COMPARISON AS KEY FEATURE 

In fact, information about the quality of programmes and institutions serves more 
and more as a means of comparison of programmes, rather than a means of providing 
information about single programmes or institutions. Naturally, this has an impact on 
the type and scope of the information to be gathered. Currently, in order to compare 
information, one is often offered quantitative data. 

INFORMATION FOR DEVELOPMENT AS KEY FEATURE 

In an increasingly competitive and diversified higher education system, the meaning 
and relevance of quality as such changes: Quality in higher education is not only 
the result of the intrinsic motivation of teachers and researchers, nor is it to be 
demonstrated as a matter of accountability. In a highly competitive field, quality in 
higher education has become a core success factor for the institutional success of HEIs. 
The indicators for the performance of HEIs in this respect are well known. They are, 
among others: 

• Completion rates/placement success 

• Research output 

• Acquisition of third party funds 

• Effectiveness/efficiency in financial terms 

• Number of Nobel-prize winners/other awards. 

Thus, information about the quality of programme and institutional levels has become 
a prime source for the management decisions at programme and at institutional level. 

In conclusion, transparent and reliable information about the quality of programmes 
and institutions are preconditions for developing programmes and institutions, 
enhancing quality, and demonstrating accountability. 

2. Information about higher education: which tools provide what kind of 
information? 

2.1 THE BOLOGNA TRANSPARENCY TOOLS 

It is noteworthy that transparency in higher education and tools for creating 
transparency are not new issues in the discussion about moving the Bologna Process 
further. Quite the contrary, the Bologna Process may even be named the most 
significant factor in striving for transparency in European higher education, right from 
the beginning and throughout the whole process: 

The core aim of the Bologna process, as stipulated in the Bologna Declaration, 
demonstrates that the whole process aims at more transparency. European Ministers 
responsible for higher education agreed to strive for: 

• Adoption of a system of easily readable and comparable degrees, also through the 
implementation of the Diploma Supplement. 

• Establishment of a system of credits - such as in the ECTS system - as a proper 
means of promoting the most widespread student mobility. 


At the Berlin Ministerial conference in 2003, European Ministers responsible for 
higher education pointed to transparency as the core aim of the Bologna Process when 
they stated: 

Quote (Berlin Communique): 

“Ministers welcome the various initiatives undertaken since the Prague Higher 
Education Summit to move towards more comparability and compatibility, to make 
higher education systems more transparent and to enhance the quality of European 
higher education at institutional and national levels.” 

In addition, the Ministers named four core transparency tools: 

Quote: 

“Ministers encourage the member States to elaborate a framework of comparable 
and compatible qualifications for their higher education systems, which should 
seek to describe qualifications in terms of workload, level, learning outcomes, 
competences and profile. They also undertake to elaborate an overarching framework 
of qualifications for the European Higher Education Area. [. . .] They appeal to 
institutions and employers to make full use of the Diploma Supplement, so as to take 
advantage of the improved transparency and flexibility of the higher education degree 
systems, for fostering employability and facilitating academic recognition for further 
studies.” 

In Bergen, the Ministers highlighted transparency as one of two major underlying 
principles of the European Higher Education Area (EHEA), next to quality: 

Quote (Bergen Communique): 

“Building on the achievements so far in the Bologna Process, we wish to establish 
a European Higher Education Area based on the principles of quality and 
transparency.” 

In London, the Ministers stressed again the role of Qualifications Frameworks for 
creating transparency in the EHEA: 

Quote (London Communique): 

“Easily readable and comparable degrees and accessible information on educational 
systems and qualifications frameworks are prerequisites for citizens’ mobility and 
ensuring the continuing attractiveness and competitiveness of the EHEA. [. . .] 
Qualifications frameworks are important instruments in achieving comparability 
and transparency within the EHEA and facilitating the movement of learners within, 
as well as between, higher education systems. They should also help HEIs to develop 
modules and study programmes based on learning outcomes and credits, and improve 
the recognition of qualifications as well as all forms of prior learning.” 


In Leuven, the role of Diploma Supplement and ECTS as important transparency tools 
were reconfirmed: 

Quote (Leuven/Louvain-la-Neuve Communique): 

“Moreover, the Bologna Process has promoted the Diploma Supplement and the 
European Credit Transfer and Accumulation System to further increase transparency 
and recognition 

These tools provide information about the type and profile of a programme or 
a qualification. They help (prospective) students, employers and public at large 
to understand what can be expected from a graduate having obtained a certain 
qualification. This information may serve as a basis for the choice of a programme or in 
the recruitment processes. In addition, these tools foster transparency at a system level 
as far as comparison of certain higher education systems is concerned. 

2.2 Rankings, classifications and transparency 

The current discussion in Europe about transparency tools is still basically a discussion 
about rankings. Apart from rankings, another initiative may be grouped under this 
term: the initiative towards “A Classification on Higher Education” in Europe, funded 
by the European Commission. 

RANKINGS 

Rankings list certain groups of HEIs that are ranked comparatively according to 
common numerical performance indicators. A major feature of this way of describing 
(and comparing) the quality of an institution is the fact, that the information which 
the ranking provides is not generated by the institution, but rather, by statistical data. 
Rankings do not refer to and do not analyse institutional processes. So, they may 
ignore the causes for high or poor performance. Thus, rankings do not directly derive 
recommendations from their findings. The purpose of rankings may be described as 
follows: 

Providing a comparative view at the system level helps to understand the 
performance of a whole higher education system and of single institutions compared 
to others. By doing so, rankings partly support the accountability function of 
HEIs, because they provide the public with information about aspects of quality of 
a programme or an institution. Not least because they are mainly organised and 
published by magazines that often serve as a guide to HEIs. 

Although the scope of rankings may differ from national or regional to global scale, 
the methodologies are similar: Quantitative data on a set of indicators is collected by 
third parties, then scored as regards each indicator and finally weighed and aggregated. 
More recently, as a consequence of the critics, the emergence of multidimensional 
rankings, which use several types of indicators to demonstrate that the quality of a HEI 
has various dimensions depending on the viewpoint of the stakeholder. 

Today, the so called “Berlin principles”, which were published in 2006 by CHE, 
UNESCO-CEPES and IHEP, form a consensus of underlying principles for the design 
of rankings. They comprise standards and principles for purpose and goals, design and 
weighting of indicators, collecting and processing data, and finally, presentation of 
ranking results. They specifically call for: 


• Transparency and clarity in terms of purpose, context, and used data 

• Multidimensional approaches 

• Outcome orientation 

• Quality assurance measures to ensure validity of data 

CLASSIFICATION 

The project called Classifying European Institutions for Higher Education (CEIHE) 2 , 
carried out by the Center for Higher Education Policy Studies (CHEPS) at the 
University of Twente, tries to develop a tool to describe the increasing diversity in 
European higher education. This tool shall enable stakeholders to see the various 
missions and profiles of HEIs. 

The classification project uses empirical data as its main methodological feature, 
which distinguishes it from typologies that address conceptual entities. The 
classification shall be multi-dimensional, non-hierarchical and descriptive instead of 
prescriptive. 

The information provided by rankings and the classification project relates to current 
performance concerning indicators and to the profile of an institution respectively. 

In other words, classifications describe horizontal diversity and provide institutional 
profiles while rankings assess vertical diversity within types of institutions and result in 
performance-based outcome. In doing so, these tools may provide relevant information 
for a better understanding of the very nature or the current state of a higher education 
system. This information may serve as a basis for the formulation of policies. In 
particular, rankings may also inform funding decisions as far as allocation of funds is 
performance driven. 

2.3 Quality assurance and transparency 

External quality assurance procedures may cover a wide spectrum of processes 
designed to monitor, maintain and enhance quality. These are basically evaluations, 
audits and accreditations at programme and institutional levels. In principle, quality 
assurance serves two main purposes: enhancement and accountability. Internal quality 
assurance naturally focuses on the enhancement of quality in teaching and learning, 
while external quality assurance, at its best, both serves the needs for accountability 
of institutions to stakeholders and the wider public, and plays a developmental role for 
enhancing quality in institutions. 

This understanding is based on the core principle of quality assurance in the 
EHEA, whereby the primary responsibility for quality rests with the Higher Education 
Institutions. 

Quality assurance in higher education as such and the design of the procedures are 
based on the Standards and Guidelines for Quality Assurance in the European Higher 
Education Area (ESG) which were developed by the E4 group. The main purpose of 
these standards and guidelines was to guarantee professionally conducted quality 
assurance procedures on a high quality level. The ESG prefer the generic principle to 
the specific requirement, focusing more on what should be done than how it should be 
achieved. The ESG combine two aspects in particular: 


2 http://www.utwente.nl/mb/cheps/research/current_projects/Classifying%20European%20Institutions%20for%20higher%20 
education. doc/ 


• Common standards for professionalism in terms of procedures and agencies 
that are no longer shared only within Europe and the EHEA, but increasingly 
worldwide; 

• The European notion of quality assurance is laid down in the following principles: 

- The primary responsibility for quality lies with the institutions. 

- Internal quality assurance forms the basis of the whole quality assurance 
system. External quality assurance has to take into account the results of the 
internal part. 

- Quality assurance processes, irrespective of the very nature and design of 
the chosen approach, have to serve the developmental function of quality 
assurance. 

- Involvement of all relevant stakeholders, including students and employers, in 
quality assurance processes and quality assurance agencies. 

As a result of the application of the ESG in quality assurance systems and processes 
in the participating countries of the Bologna Process, quality assurance in the EHEA, 
while respecting the principle of subsidiarity, is based on a common ground of shared 
values and principles and is conducted in the specific European way of professionally 
sound processes with the participation of all stakeholders. The application of the ESG is 
a major achievement of the Bologna Process. 

The purpose and role of quality assurance are affected by the growing competition 
and diversification in higher education. If quality in higher education is a core 
success factor for institutional success, the quality assurance processes should not 
be considered as something “extra” in addition to the other core processes in higher 
education, but should be integral. Guaranteeing a certain quality or enhancing the 
quality of a programme becomes an integral part of the regular management of a HEI. 
There is a trend of moving from quality assurance towards quality oriented higher 
education management, which means that former quality assurance processes get 
directly linked to core management processes. 

Hence, quality assurance has to take these differences into account, and has to take 
as its point of departure the mission and the objectives of a specific institution and 
recommend improvement to achieve the set goals. This calls either for process designs 
which are generic in order to be able to be applied to HEIs of different profiles etc, or 
for the diversification of quality assurance processes in order to match the exact needs 
of the given HEI. 

Two major features of the type of information that is provided by quality assurance 
are: 

• The link between processes and performance/quality; 

• The link between evaluation of quality and performance (including weaknesses 
and strengths) and recommendations for future actions. 

One can say that quality assurance not only provides comprehensive information about 
strengths and weaknesses and thus a holistic view on the quality of a programme or an 
institution, it also draws the attention to potential future enhancement. By this means, 
quality assurance generates necessary information for the management of HEIs and 
for their further development. It is fair to say that the extent to which the outcomes 
of quality assurance relate to recommendations for the future depends on the type 


of procedure. It may be higher in case of classical evaluations and lower in case of 
accreditation processes. 

ENQA is well aware of the fact that currently, different actors and stakeholders 
in higher education weigh the two main purposes differently. The accountability 
dimension becomes prominent especially in times of financial constraints and a drift 
towards more autonomy for HEIs. However, rather than reducing quality assurance 
to one of the two, ENQA supports the development of quality assurance processes by 
maintaining a close link between both dimensions, which is clearly to the benefit of the 
HEI. There is a concern that if HEIs are forced to focus on rankings or transparency 
tools, they will gear their efforts to this end, rather than striving to build a true quality 
culture encompassing both assurance and enhancement. 

3. Conclusion 

TRANSPARENCY: DIFFERENT TYPES OF INFORMATION ARE NECESSARY 
FOR DIFFERENT PURPOSES 

In the current discussion about the relationship between rankings and quality 
assurance, the notion of “detailed and reliable information on the quality of individual 
study programmes, faculties and higher education institutions” claims that participants 
in the discussion talk about the same issue. Actually, this is not the case. 

Firstly, what is quality in higher education? Do rankings and quality assurance really 
use the same concepts of quality for a programme or an institution? 

Secondly, what is information? Is the type of information provided by rankings really 
the same as the type of information generated by quality assurance? 

ABOUT QUALITY IN HIGHER EDUCATION 

Rankings/classifications provide comparative performance information about certain 
aspects of a programme or an institution (i.e. completion rates of students and 
graduates, or number of Nobel prize-winners among staff). The information that 
is gathered through ranking contains rarely information about the quality of the 
programme or the institution. Depending on the type of ranking, various performances 
are weighed in order to draw a more comprehensive picture. 

Quality assurance provides comprehensive information about the compliance of 
a programme or an institution with certain standards and about the strengths and 
weaknesses of the programme or the institution. Normally, quality assurance is not 
restricted to certain aspects but based on a holistic view. 

Thus, the underlying concepts of quality in higher education differ a lot between 
rankings/classifications and quality assurance. 

ABOUT INFORMATION 

In the case of rankings/classifications, information consists of aggregated data, 
which may combine and weigh data referring to different aspects of performance of 
programmes or institutions at a certain point of time. 

In the case of quality assurance, information consists of performance, internal 
processes, causes for strengths and weaknesses or compliance/non-compliance of a 
programme or an institution with certain standards, combined with recommendations 
for future developments. Thus, also regarding the production of information, quality 
assurance has to serve two functions: information for the sake of accountability and 


development. ENQA members will, in line with the ESG, remain mindful that reports 
arising from quality reviews should be fit for purpose and be accessible to various 
audiences. Therefore, internal or external review reports should be supplemented by an 
executive summary to ensure that it is accessible to non specialist readers. 

ABOUT PRODUCING INFORMATION 

When the requested information serves comparison purposes, it has an impact on 
the type and scope of information. Thus, in the case of rankings and classifications, 
information needs to be easily compared, which normally means quantitative data. One 
needs standardised information that can be equally accessed by all institutions. 

When the requested information (also or partly) serves strategic and developmental 
needs of the institutions, as is the case in quality assurance, one needs information 
generated for the individual case. This goes beyond aggregating quantitative data and is 
normally done by a well-informed peer review. 

In addition, the current discussion about the relationship between rankings and 
quality assurance does not take into account the role and meaning of transparency 
tools at the programme level, such as qualifications frameworks, diploma supplements 
and ECTS. These Bologna transparency tools play a crucial role as regards providing 
transparent and reliable information about single programmes. 

Conclusion: 

Bologna transparency tools, quality assurance and rankings/classifications 
serve different but partly overlapping purposes, use different methodologies, 
provide different types of information. 

Bologna transparency tools, quality assurance and rankings/classifications serve 
different purposes, although they support each other to a certain extent. It is 
misleading to consider them as alternatives. 

From the view point of quality assurance and its two main purposes, accountability 
and enhancement, their relationship to other transparency tools looks as follows: 

The Bologna transparency tools are important reference points for quality assurance. 
The qualification framework for the European Higher Education Area (EHEA) 
describes the level and scope of qualifications that graduates shall have acquire by the 
time of graduation. The diploma supplement describes the profile of the qualification, 
and ECTS is an important tool to guarantee a realistic curriculum design as regards 
student workload, and it also fosters mobility. 

By putting performance of institutions in relation to criteria, rankings and 
classifications contribute to the accountability function by informing the public, 
but they do not contribute to quality enhancement, the second function of quality 
assurance. Thus, ranking/classification tools are not to be seen as quality assurance 
tools; in particular, they do not provide information about potentials for the future. 
They might be seen rather as providers of a certain type of information that is useful 
for quality assurance. On the other hand, it is true that quality assurance does not 
necessarily provide quantitative information on aspects of the performance of a 
programme or an institution for comparison purposes. 

ENQA wishes to assert that it is of utmost importance for the development of trust 
that all activities related to the production of information about higher education in the 
European Higher Education Area (EHEA) have explicit goals and methodologies. These 


must be communicated transparently in order to avoid misinterpretations of purposes 
and outcomes of the procedures. This should concern all types of providers, whatever 
their status is, and should notably be true with regard to: 

• Information, recommendations or accreditation outcomes used by decision 
makers in charge of steering higher education policies; 

• Information and guidance for students; 

• Evaluations and recommendations devoted to help higher education institutions 
and programmes in their continuous improvement strategies and actions; 

• Benchmarking or rankings developed for the purpose of external communication. 

All activities related to quality assurance in higher education or production of 
information about the quality of higher education should respect the fitness for purpose 
principle. It is crucial that the purpose(s) should determine the types of scrutiny, 
methodology and procedures applied. Decisions on the type of scrutiny, methodology 
and procedures to be applied should be preceded by, and based on, the identification of 
purpose(s). 

Therefore, the discussion about the pros and cons of quality assurance and 
transparency tools for better information of higher education institutions and 
stakeholders should not mix up both approaches, which, due to different purposes, 
methodologies and outcomes, do not compete with each other. 

Rankings and classifications are restricted to the accountability function, whereas 
quality assurance will always serve, in different ways, both purposes. The two 
approaches overlap with the accountability function and may thus be complementary. 
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ENQA seminar "Quality and Transparency in Higher Education: Expectations, 
Tools and the Link to Institutional Autonomy" 

17-18 May 2010 

Hosted by Magna Charta Observatory of Fundamental University Values and Rights 


Venue: 

Aula Giorgio Prodi 
San Giovanni in Monte 
Piazza San Giovanni in Monte 2 
40124 Bologna 


PROGRAMME 
Monday 17 May 2010 

9.30 - 10.00 Welcome statement and Setting the stage 

Achim Hopbach, President of ENQA 

Bastian Baumann, Secretary General of the Magna Charta 
Observatory 

10.00 - 12.00 HE and Quality: Expectations, Quality Standards and Tools. 

What kind of information are we looking for? 

And for what purpose? 

Inputs from: 

- HEIs: Tia Loukkola, EUA; 

Stefan Delplace, EURASHE; 

Carla Salvaterra, Vice Rector for International Relations, 
University of Bologna 

- Students: Allan Pall, ESU 

- Employers: Marita Aho, Business Europe 

- Policy-makers: Andrejs Rauhvargers, BFUG 
Discussion 

Chair: Padraig Walsh, IUQB 

12.00 - 13.00 Lunch 


13.00 


14.30 

15.00 


17.00 

17.30 

20.00 


14.30 HE and Quality: Expectations, Quality Standards and Tools. 
What kind of information do we provide? And for what purpose? 

Inputs from: 

- ENQA: Karl Dittrich, ENQA Board member 

- QA agencies: Gyorgy Bazsa, HAC and Helka Kekalainen, FINHEEC 

- EUA: Tia Loukkola 

Chair: Patrick Van den Bosch, VLIR 

15.00 Coffee Break 

17.00 HE and Quality: Expectations, Quality Standards and Tools. 
What kind of information do we provide? And for what purpose? 

Introduction by Robin Van IJperen, European Commission 

Inputs from: 

- Classification project (CEIHE): David Bohmert 

- Rankings: Gero Federkeil, CHE 

Discussion 

Chair: Robin Van IJperen, European Commission 

17.30 Wrap-up of the First day, Radu Damian, ARACIS 

End of the first day 

Dinner at restaurant Pappagallo 

Piazza della Mercanzia 3/c - 40125 Bologna 


Tuesday 18 May 2010 


09.00 - 11.00 

Influence of rankings and other transparency tools on HEI 
autonomy 

University autonomy and accountability: where is the balance? 

Inputs from: 

- Helene Lamicq, Magna Charta Observatory 
Responses 

- Janja Komljenovic, BFUG 

- Achim Hopbach, ENQA 

- Gero Federkeil, CHE 

Chair: Bastian Baumann, Magna Charta Observatory 

11.00 - 11.30 

Coffee break 

11.30 - 13.00 

Final discussion: 

Purpose and standards for information tools on quality in HE 

Input statements by all stakeholders 

Chair: Bastian Baumann, Magna Charta Observatory 

13.00 

Conclusions 

Achim Hopbach, ENQA 


Lunch 



The present report is based on the contents and results of an ENQA seminar on 
quality assurance and transparency tools, hosted by Magna Charta Observatory of 
Fundamental University Values and Rights on 17-18 May 2010 in Bologna, Italy. 
The purpose of this seminar was to discuss expectations from HEIs and other 
stakeholders concerning information on quality of HE and the contribution of 
quality assurance and transparency tools to meet these expectations. 



